1. Field of the Invention
The present invention relates to a texture processing apparatus, method and program for high-quality texture mapping techniques in the field of calculator graphics (CG).
2. Description of the Related Art
In recent years, CG techniques have rapidly been progressed, and realistic graphics renderings that have a strong resemblance to photographs have come to be possible. However, many high-quality CG products for movies and TV programs cost extremely high, since they are often produced by long-time hard-working manual efforts. In the future, there will be an increasing demand for various CG renderings, and hence for techniques for easily producing high-quality CG produces at low cost.
A texture mapping technique is a technique for rendering the surface texture of a CG model, and is widely used for CG image production. Specifically, it is used to attach, to the surface of a CG model, images obtained by photography, or images drawn by a content producer, based on texture coordinates prepared for CG data. In rendering the texture of a CG model surface, it is considered especially difficult to render cloth, skin or fur. To render a material of a soft feel, it is very important to express changes in the surface color of a material or changes in the self shadow of the material, which depend upon a direction (i.e., a viewpoint direction) in which the material is viewed and a direction (i.e., a light direction) in which illumination is applied.
In view of the above, a scheme for photographing real materials and reproducing the properties of the materials to generate realistic CG images has come to be actively used.
Concerning rendering of the texture of a surface corresponding to viewpoint and light directions, researches of modeling schemes called, for example, Bidirectional Reference Distribution Function (BRDF), Bi-directional Texture Function (BTF) and Polynomial Texture Maps (PTM) have been advanced. These schemes employ approaches to analyze acquired data to derive a function model. Further, an approach to hold, as texture data, an image obtained by photography, and then to adaptively select and map textures based on parameters, such as the viewpoint direction and light direction, has been proposed by, for example, JP-A 2006-146326 (KOKAI). These approaches enable realistic texture rendering of materials that are hard to model, i.e., materials that involve complex color/luminance changes.
On the other hand, media for providing such CG images as the above have also rapidly been advanced. For instance, three-dimensional or stereoscopic display devices of various schemes capable of displaying three-dimensional moving pictures have recently been developed. In particular, there is an increasing demand for a flat panel type display that does not require, for example, dedicated glasses. A scheme of providing, immediately before a display panel (display device) with pixels fixed in position, such as a direct-view or projection type liquid crystal display device or plasma display device, a ray control element used to direct, to an observer, rays emitted from the display panel is known as a scheme that can be relatively easily realized.
The ray control element is generally called a parallax barrier, and is used to control light so as to enable an observer to see different images at different angles even at the same position on the ray control element. Specifically, when only a horizontal parallax is imparted, a slit or lenticular sheet (cylindrical lens array) is used. When both horizontal and vertical parallaxes are imparted, a pin-hole array or lens array is used. The scheme utilizing a parallax barrier is further classified into a binocular scheme, multi-view scheme, super multi-view scheme (a super multi-view condition for a multi-view scheme), and integral photography (hereinafter referred to as “IP”). The basic principle of these schemes is substantially the same as that employed for stereo photography and invented about 100 years ago.
In both the IP scheme and multi-view scheme, the visual distance is usually limited, and hence a display image is formed as if a perspective projection image at the visual distance was actually seen. In the case of the IP scheme (one-dimensional IP scheme) in which only a horizontal parallax is imparted (i.e., no vertical parallax is imparted), if the horizontal pitch of the parallax barrier is an integer multiple of the horizontal pitch of pixels, there exist a plurality of parallel rays. In this case, a correctly projected stereoscopic image can be obtained by dividing, in units of pixel lines, an image obtained by performing perspective projection of a constant viewing distance in the vertical direction and performing parallel projection in the horizontal direction, and superimposing the resultant image components into a parallax synthesis image as an image format for displaying an image on a display surface (see, for example, JP-A 2005-086414 (KOKAI)). In the multi-view scheme, a correctly projected stereoscopic image can be obtained by distributing images acquired by simple perspective projection.
Further, an imaging apparatus, in which different projection methods or different projection center distances are employed in the vertical direction and horizontal direction, must employ a camera or lens of the same size as that of a subject when, in particular, parallel projection is performed. Therefore, it is hard to realize such an apparatus. This being so, to obtain parallel projection data by photography, the use of a method for converting data obtained by perspective projection is more practical. This method includes, for instance, a known ray spacing method which utilizes interpolation using EPI (epipolar plane).
The above-mentioned techniques are background techniques for enhancing the texture of a material in CG, and for realizing stereoscopic vision on a stereoscopic display. However, there is a case where a texture or stereoscopic effect that satisfies a user may not be obtained simply by mapping texture data corresponding to material images obtained by photography. This is because there are some constraints (specifications) on the color representation power of a medium or the depth representation power of a stereoscopic display. Accordingly, there is a demand for a technique of emphasizing texture data before texture mapping processing, thereby enhancing the texture of a material in CG or the stereoscopic effect of a stereoscopic display.
When a plurality of sheets of texture data acquired under different viewpoint conditions and light conditions are directly mapped onto a CG model, the resultant image may not have quality that satisfies a user, depending upon a medium that presents the image. This is because there are some constraints (specifications) on the color representation power of a medium for presenting CG images, or on the depth representation power of a stereoscopic display.
In the case of processing a single texture data item, the texture of a CG image surface can be emphasized by, for example, subjecting the texture data to image processing to thereby enhance its contrast. However, in the case of processing a plurality of texture data items, any means for emphasizing the texture of a CG image surface has not yet been established for a method of selecting a to-be-mapped data item from the texture data items, or a method of processing the selected texture data item.